Skip to content

fix(server): keep a failed service start out of the current state - #12199

Open
vitalyiegorov wants to merge 1 commit into
pingdotgg:mainfrom
vitalyiegorov:fix/launchd-bootout-wait
Open

vitalyiegorov wants to merge 1 commit into
pingdotgg:mainfrom
vitalyiegorov:fix/launchd-bootout-wait

Conversation

@vitalyiegorov

@vitalyiegorov vitalyiegorov commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Fixes #12197. On macOS 15 the first t3 service install can fail at launchctl bootstrap (#11995). By then the plist, service-state.json and runtime already name the new version, and nothing records the failed start. The retry prints T3 Code service is already installed and exits 0 while launchd keeps running the previous launcher, so every remote update stays blocked by the #11940 protocol gate.

Change

BootService.install in apps/server/src/cloud/bootService.ts now writes the existing .restart-pending marker on every install, not only for start: false. It's one removed conditional. A successful activate still removes the marker, and so do restart and the launcher when it comes up on that version. A failed start therefore leaves status reporting restart-pending, and the next install repairs the service instead of calling itself current.

This complements #12005, which makes the stop and start actually replace the job. This PR only makes the retry honest.

Scope and approval

Julius triaged #12197 as a real bug, separate from #11995, and labelled it accepted: #12197 (comment)

Verification

  • New test: on launchd, a started install whose launchctl bootstrap fails leaves the marker with the CLI version. status then reports problems: ["restart-pending"] with current: false. A second install (what reconcileService does for a status that isn't current) runs bootout, enable and bootstrap again and removes the marker.
  • The systemd install test also asserts that a successful started install leaves no marker.
  • vp test run apps/server/src/cloud/bootService.test.ts: 39 passed, after rebasing on current main. Server typecheck, plus lint and format on the touched files.
  • Reproduced and diagnosed on a Mac Studio running macOS 15.7.4, where the old npm launcher was still loaded under the new plist.
  • No UI changes.

Implemented with Claude Fable 5.1 and Claude Opus 5 in T3 Code (Claude Code harness).

🤖 Generated with Claude Code

@github-actions github-actions Bot added size:S 10-29 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Sep 17, 2026
@macroscopeapp

macroscopeapp Bot commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at 2a55165

Macroscope's review found this PR approvable — This is a narrow service-install recovery fix that preserves the marker when activation fails, preventing a failed start from being reported as current and enabling a later retry. Successful installs still remove the marker, and targeted regression tests cover the failure and recovery flow.

No code changes detected at 0905300. Prior analysis still applies.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Path: .coderabbit.config.ts
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c9347fd8-d16d-41a6-bb57-18e2da15af04
📥 Commits

Reviewing files that changed from the base of the PR and between de4a8db and 0905300.

📒 Files selected for processing (2)
  • apps/server/src/cloud/bootService.test.ts
  • apps/server/src/cloud/bootService.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Every install now writes a restart-pending marker before updating service state or the unit. Successful activation removes the marker. macOS tests verify that failed startup leaves the service non-current and that a later install retries activation and clears the marker.

Changes

Restart-pending recovery

Layer / File(s) Summary
Install marker and recovery flow
apps/server/src/cloud/bootService.ts, apps/server/src/cloud/bootService.test.ts
install() writes the restart-pending marker before service updates. Failed launch-agent startup leaves the marker, and status reports restart-pending and not current. A successful retry removes the marker and reports the service as current.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Sequence Diagram(s)

sequenceDiagram
  participant BootService.install
  participant launchctl
  participant BootService.status
  BootService.install->>BootService.install: Write restart-pending marker
  BootService.install->>launchctl: Bootstrap launch agent
  launchctl-->>BootService.install: Startup failure
  BootService.install->>BootService.status: Check service status
  BootService.status-->>BootService.install: Report restart-pending and not current
  BootService.install->>launchctl: Retry bootout, enable, and bootstrap
  launchctl-->>BootService.install: Startup succeeds
  BootService.install->>BootService.install: Remove restart-pending marker
Loading

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to 09053

The failed-start retry now reports the service as not current and retries activation. No merge-blocking issue is evident after normal checks.

Architecture Summary

Architecture risk: 🔵 Low · up to 09053

The change affects 1 system.

Changed systems: apps/server

Architecture concerns
No architecture-level concerns identified.

Review details

Systems and components

  • observed — apps/server (service) was modified; 2 changed files map to changed impact.

Before / after behavior

  • observed — Modified behavior in apps/server/src/cloud/bootService.test.ts: The successful-install test now checks that the restart-pending marker does not exist after installation completes.
  • observed — Modified behavior in apps/server/src/cloud/bootService.test.ts: Adds a macOS test where a failed launch-agent bootstrap makes installation fail with BootServiceCommandError, leaves a marker containing 1.2.3\n, and causes status to report restart-pending and not current. After clearing the simulated failure, a subsequent install retries bootout, enable, and bootstrap; the test verifies the marker is removed and status is current.
  • observed — Modified behavior in apps/server/src/cloud/bootService.ts: Install now writes the restart-pending marker on every install, before updating service state or the unit. Previously, the marker was written only when an installed service was updated with start: false; started installs remove it after activation, while failures before removal leave it present.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: preventing a failed service start from being reported as current.
Description check ✅ Passed The description covers the problem, change, scope and approval, and verification. It includes the linked issue, focused test results, and environment details.
Linked Issues check ✅ Passed For #12197, every install now writes .restart-pending with the target version. The reported tests verify that failed launchd bootstrap leaves the marker, status reports restart-pending and `curr…
Out of Scope Changes check ✅ Passed The production change and tests cover restart-pending tracking and retry behavior for #12197. The reported successful-install marker check also supports this behavior. No unrelated change is indicated…
Approvability ✅ Passed This is a focused bug fix in apps/server/src/cloud/bootService.ts, with regression tests in apps/server/src/cloud/bootService.test.ts. Install now writes the existing .restart-pending marker bef…
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR

Comment @coderabbitai help to get the list of available commands.

@vitalyiegorov
vitalyiegorov force-pushed the fix/launchd-bootout-wait branch 3 times, most recently from cb1e33b to de4a8db Compare September 17, 2026 15:55
@juliusmarminge juliusmarminge added the macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews label Oct 1, 2026 — with ChatGPT Codex Connector
@vitalyiegorov
vitalyiegorov force-pushed the fix/launchd-bootout-wait branch 3 times, most recently from e8751ed to cb709a8 Compare October 6, 2026 03:28
On launchd, `t3 service install` writes the plist and state before
`launchctl bootstrap`. When bootstrap fails, nothing on disk records it,
so the next `t3 service install` reports "already installed" while the
previous launcher keeps running.

Write the existing `.restart-pending` marker on every install, not only
for `start: false`. A successful start still removes it; a failed one now
leaves `status` reporting `restart-pending`, so the retry repairs the
service.

Fixes pingdotgg#12197

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@vitalyiegorov
vitalyiegorov force-pushed the fix/launchd-bootout-wait branch from cb709a8 to 0905300 Compare October 7, 2026 12:27

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

macroscope-review Opt PRs made by unvouched contributors in for Macroscope review. Vouched contributors auto-reviews size:S 10-29 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: macOS t3 service install reports "already installed" while launchd still runs the previous launcher

2 participants